Family Physicians and Youth Tobacco-free Education: Outcomes of the Colorado Tar Wars Program
http://www.100md.com
《家庭医学美国版医学期刊》
Department of Family Medicine, University of Colorado Health Sciences Center, Denver, CO
Department of Family Medicine, The Children’s Hospital, Denver, CO
Abstract
Background: Tar Wars is a national school-based tobacco-free education program operated by the American Academy of Family Physicians. The Tar Wars lesson uses an interactive 45-min session taught by volunteer family physicians in 4th- and 5th-grade classrooms and focuses on the short-term image-based consequences of tobacco use. In this study, we evaluated the effectiveness of the Tar Wars program in Colorado with both quantitative and qualitative measures.
Methods: Students participating in the quantitative evaluation were tested before and after a Tar Wars teaching session using a 14-question test covering the short-term and image-based consequences of tobacco use, cost of smoking, tobacco advertising, and social norms of tobacco use. Qualitative evaluation of the program included guided telephone interviews and focus groups with participating students, teachers, and presenters.
Results: Quantitative evaluation showed statistically significant improvement in correct responses for the 14 questions measured with an average increase in correct responses from 8.95 to 10.23. Three areas recommended by the Centers for Disease Control (CDC) for youth tobacco prevention showed greater change in correct responses, including cost of smoking, truth of tobacco advertising, and peer norms of tobacco use. Qualitative evaluation found that the overall message of the session was well received, that previously known tobacco information was reinforced by its presentation in a novel format, and that new information learned included cost of smoking, truth of tobacco advertising, and peer norms of tobacco use.
Conclusions: The Tar Wars lesson plan is effective in increasing students’ understanding about the short-term consequences of tobacco use, cost of tobacco use, truth of tobacco advertising, and peer norms. Tar Wars meets the CDC guidelines as one component of effective comprehensive youth tobacco prevention.
Tobacco remains the no. 1 preventable cause of death in the United States, causing >400,000 deaths annually.1 Each day in the United States, approximately 4400 youths aged 12 to 17 years try their first cigarette,2 with 21.9% of US high school students using tobacco regularly.3 Nine of 10 current adult smokers started their habit before the age of 19 years.4 Major influences on youth tobacco initiation continue to be tobacco advertising5–7 and social norms.8
The 2000 Surgeon General’s Report recommends comprehensive tobacco prevention activities to reduce tobacco use, including community interventions that include school-based education.1 Because many students start using tobacco before high school, the Centers for Disease Control (CDC) Best Practices recommend that tobacco use prevention begin in elementary school and continue through middle and high school.9 The CDC Department of Adolescent and School Health’s Tobacco Prevention Guidelines specifically recommend school-based instruction about the short- and long-term negative physiologic and social consequences of tobacco use, social influences on tobacco use, peer norms regarding tobacco use, and refusal skills.10 Instructional programs using these components have been shown to significantly reduce smoking prevalence for up to 5 years after program completion.11, 12
Comprehensive statewide tobacco prevention programs have been shown to effectively reduce youth tobacco use13–19 and have shown school-based programs to be an effective component of statewide tobacco control.20, 21 Meta-analysis of youth tobacco prevention programs suggests that those programs focusing on social influence resistance models are more effective than programs designed to arouse concern or fear by focusing on death and other health risks.22
Tar Wars is an in-school, tobacco-free education program owned and operated by the American Academy of Family Physicians (AAFP). Tar Wars was designed in 1988 in Denver, CO, by Doctors Ought to Care (DOC) and the Hall of Life at the Denver Museum of Nature and Science.23 Tar Wars ownership was acquired by the AAFP in 2000, is currently taught in 50 states and 14 countries, and has reached 8 million children. Tar Wars is a lesson plan intended to be one component in a comprehensive approach to youth tobacco prevention. Previously published literature reports on Tar Wars have included descriptions of the program23, 24 and program perspective of involved students, teachers, and presenters.25 This study provides a qualitative and quantitative examination of the impact and effects of the Tar Wars program in Colorado elementary schools.
Methods
During 2001 and 2002, the Colorado Tar Wars program was operated by the Colorado Academy of Family Physicians Foundation (CAFPF) with a grant from the Colorado Department of Public Health and Environment, State Tobacco Education and Prevention Partnership (STEPP). Researchers from the Department of Family Medicine at the University of Colorado Health Sciences Center independent of the CAFPF and Tar Wars conducted this evaluation of the program, which was approved by STEPP. Because of constraints of the STEPP grant, the study was limited to implementation and evaluation of the Tar Wars program during one academic year.
This study evaluates the Tar Wars lesson program as implemented in Colorado during the 2001–2002 academic year. The Tar Wars lesson plan evaluated is offered free through the AAFP Web site (www.tarwars.org) and includes 3 components: a preactivity given by the classroom teacher in the days before the guest speaker, a classroom session led by volunteer family physicians and health educators, and the poster contest coordinated by the class teacher. The preactivity’s objective is to teach students that, contrary to their perception, tobacco users are in the minority. The classroom session includes 5 activities: short-term effects of tobacco use (bad breath, smelly clothes, yellow teeth, coughing, ashes, etc), demonstration of the physical effect of decreased lung capacity (breathing-through-straw exercise), cost of tobacco use (per week, month, year), reasons smokers give for tobacco use (contrasted with actual short-term effects), and an analysis of tobacco advertising. The poster contest challenges students to illustrate their integration of the Tar Wars lessons by the creation of a poster showing the positive effects of not using tobacco.
Schools were recruited by mailing invitations to participate in Tar Wars to all 5th-grade classrooms in the state. Family physicians were recruited by a mailing sent to all members of the CAFP. Schools were matched geographically with family physicians by the CAFPF Tar Wars staff. The evaluation included the first 68 schools matched with speakers, with the goal of approaching 3000 students. Speaker and teacher training consisted of providing each with copies of the Tar Wars program guide as offered by the AAFP. The Tar Wars lesson plan guides teachers and presenters on each activity, including an objective, time requirement, suggested speaker comments, and activity instructions. The session is designed to be interactive with a high level of student participation.
For the quantitative portion of this study, surveys containing 14 questions regarding knowledge about smoking were administered before and after the Tar Wars session to students of 68 elementary schools in Colorado. Copies of the pre- and post-tests are included in Figures 1 and 2. Questions directly relating to lesson plan content were obtained from the AAFP’s unpublished Tar Wars survey, part of the package of Tar Wars materials. Minor changes to the questions were made by the CAFPF Tar Wars Committee and the Department of Family Medicine for clarity and to better reflect the content of the session. Fifth grade students ranging in age from 9 to 13 years completed the pre- and post-tests. This analysis attempts to determine whether there were any changes in knowledge about tobacco on the responses to the pre- and post-tests. For comparison purposes, all questions from the pretest were also asked on the post-test. Pretests were administered approximately 1 week before the intervention, and post-tests were administered 2 to 4 days after the Tar Wars presentation. To preserve student confidentiality, unique identifiers were not used in the data collection. As a result, comparisons over time could not be made at the individual student level, but school identification was retained, allowing comparisons over time across the school groups.
The 2 test of association was used to determine demographic differences between the pre- and post-test respondent population. Because multiple 2 tests were performed, statistical significance was determined at the level of 0.01. Assessment of differences between correct responses to the 14 items and average composite score (pre- and post-test) were performed using multi-level modeling, controlling for age, race, ethnicity, gender; whether there was a smoker in the household; and whether the respondent was currently using tobacco. The interclass correlation (ICC) was calculated to determine whether student composite scores vary between schools as well as within schools. General linear mixed models (Proc Mixed) with average composite score as the outcome were used to extend the traditional linear regression model to accommodate the hierarchical structure. Because we were unable to retain student identifiers, the study design was analyzed as a nested cross-sectional design, with students nested within schools at 2 distinct time points, adjusting for the tendency for students within the same school to respond more similarly than students in different schools. Variables that were significant in the model at the level of 0.10 were tested for possible interaction effects with the test (pre- vs post-test) variable. Current tobacco use was not considered because the total number of tobacco users in the sample was very small. Interactions significant at the level of 0.05 were retained in the model.
The qualitative evaluation used 2 primary methods of data collection: semi-structured interviews and focus group discussions. Semistructured, confidential telephone interviews were conducted with presenters and teachers who participated in Tar Wars during the 2001–2002 school year. In addition, evaluation forms filled out by presenters and classroom teachers after the presentations were reviewed qualitatively, focusing on the responses to the open-ended probes asking for suggestions for improvement in the program.
The CAFPF provided the evaluation team with lists of presenters, teachers, and youth educators for whom they had current contact information. We selected multiple names from the list to cover a wide geographic area within the state—both urban and rural—and scheduled interviews only with those who returned our calls and consented to an interview. The interviews asked about their experiences, reactions from students, challenges, and how the program could be improved. These interviews were audiotaped and transcribed.
Focus groups with students who participated in Tar Wars were designed to elicit responses around what students remembered from the presentation, what they liked or disliked, how they used or might use what they learned, and what ideas they have for helping other kids learn about tobacco. Along with the interview guide, additional probes were used to elicit more detailed responses where necessary. Because of funding and time limitations of the evaluation, the CAFPF contacted classroom teachers directly in the Denver metropolitan area to set up the student focus groups. In addition, because of the age of the students and possible negative reactions to audiotaping the interviews, the evaluation team decided not to audiotape focus groups. Both the facilitator and co-facilitator kept notes, including direct quotations when possible.
The transcripts and typed notes were analyzed using ATLAS-ti qualitative data analysis software application. The analyst used an "editing" style of analysis.26 The editing style presumes no pre-existing theories or hypotheses. Using this style, the researcher starts with a "blank slate" and allows the data to guide the discovery of evolving patterns or themes.
In general, we identified the most significant themes and issues that reflected views expressed by the key informants. The emergent themes were discussed among co-facilitators and the CAFPF staff to confirm or disconfirm their meaning and relevance. Results from paper surveys by classroom teachers were also reviewed to verify results from the teacher interviews conducted by the evaluation team. Salient themes discussed below generally reflect the experience of multiple individuals. Some views were expressed by one or several individuals that were not shared by others, yet reflect important contradictory or insightful perspectives.
Results
Quantitative Analysis
A total of 2926 students completed the pretest, and 2766 students completed the post-test and were included in this analysis. The pre- and post-Tar Wars respondent populations were not significantly different (P > .05) in terms of their demographics (Table 1). The sample of schools was diverse in terms of geographic spread across Colorado and represented metropolitan, urban/suburban, rural, and frontier communities. The demographics correspond generally to Colorado demographic data. The unconditional ICC correlation for students within schools was 0.08, with a conditional ICC of 0.06 (variance component Final Model (school): 0.2131, P < .0001), suggesting that hierarchical analysis would be necessary to account for the clustering of students within schools.
Analysis of the pre- and post-tests indicated that the tobacco knowledge of students increased from pre- to post-test, in general as well as across a wide variety of focus areas. Respondents’ average scores were significantly higher for the post-test than the pretest (P < .0001). This finding remained true after controlling for age, gender, race, ethnicity; whether the respondent lives with a tobacco user; whether the respondent is a current tobacco user; and clustering of students within schools. Students increased their knowledge score by an average of 1.28 correct answers on the 14 items after the intervention, with an average overall knowledge score increase from 8.95 correct responses before the Tar Wars session to 10.23 correct responses after the Tar Wars lesson. Knowledge scores showed an increase in correct responses in all 14 of the pre- and post-test responses (Table 2), with the proportion of correct responses showing significant increase between tests (P < .01).
Significant predictors of average knowledge scores included race (P = .0003), ethnicity (P < .0001), and current tobacco use by the respondent (P < .0001). For these predictors, knowledge scores are reduced by one-half a point (race), two-thirds of a point (ethnicity), and 1.4 points (current tobacco use). Significant interactions included test by ethnicity (P = .0007) as well as race by test (P = .0113). The average increase in tobacco knowledge score for minority students (non-white and Hispanic/Latino respondents) was lower than the increase in score of white non-Hispanic/Latino students. The average increase in knowledge score for minority students was 1.08 (Hispanic) and 1.10 (nonwhite students). White non-Hispanic students’ average knowledge scores increased approximately 1.50 points between tests.
The distribution of question responses tended to cluster around 3 categories:
Items for which a statistical significant improvement occurred, but a high percentage of respondents already knew the correct answers before the intervention: more than 75% of the population selected the correct response at pretest for each question. While there was a statistically significant improvement in the percentage of correct responses between pre- and post-tests, a ceiling effect exists.
For 7 of the 14 knowledge questions, the post-test correct responses increased between 4% and 10% in a significant way, indicating additional learning. These items included: "Smoking causes bad breath," "Smokers have yellow teeth," "Cigarettes that contain low tar are safe," "Smokers have clean-smelling clothes," "Smoking can decrease the amount of air that gets into your lungs," "People who use tobacco are cool, sexy, and popular," and "Some people use tobacco because their friends do."
Items that reflect a high degree of newly acquired tobacco knowledge: questions where the proportion of correct responses were initially low pretest and increased by 10% to 20%.
The correct responses to 3 of the fourteen knowledge questions were not as well known before the Tar Wars intervention, but were identified correctly by a larger percentage of the post-test respondent population. These questions included: "Smoking a pack of cigarettes each day for a year would cost at least several hundred dollars," "Advertisers tell the truth about tobacco use," and "What percentage of kids your age do not use tobacco every week."
Items with ambiguous change: questions where some learning may have occurred, but did not seem to be as effective across all responses.
These included "Smoking is relaxing," "Tobacco companies can use television as a way of advertising," and "G-rated movies are used by tobacco companies to sell tobacco."
For each of these questions there was a statistical association between pre- and post-test responses. However, even though response increases occurred between the correct responses, increases also occurred with either the incorrect responses or the "don’t know" responses.
Qualitative Analysis
We completed semistructured individual interviews (10–20 min in length) with 9 presenters, 8 teachers, and 3 youth educators. We completed 5 focus group discussions, with group sizes ranging from 7 to 15 participants. By the completion of the 5 focus group discussions, it seemed that saturation had been achieved, with few if any new topics of substance arising.
Students, presenters, and teachers whom we interviewed agreed that they liked the Tar Wars presentations and activities and that new learning occurred. Presenters and teachers reported that students were attentive and interested and reacted positively to the information, presentation style, and having someone from outside the school (especially physicians) present the information. They thought that the content and length were appropriate and that the use of hands-on activities and visual aids was effective at engaging the students. The teachers whom we interviewed agreed that Tar Wars integrates well with other health education or science lessons.
The students responded most strongly to the central activities in the lesson plan and recalled the information about tobacco use: advertising, effects on physical appearance, percentages of those who smoke, and the difficulty of breathing for smokers. The students thought that the Tar Wars program presented a different kind of message about tobacco than some of the other programs that they had experienced. For many students, the information about the cost of smoking, the percentage of youth and adults who smoke, and how tobacco is advertised also was new information.
Most students, presenters, and teachers acknowledged that students already knew that using tobacco is harmful, with long-term health effects such as cancer. However, they thought that the details about the short-term effects and other information about tobacco (eg, cost, ingredients, and advertising) were new information that helped them to understand why they should not use tobacco. Many students emphasized the importance of hearing these messages repeatedly and suggested having various sessions regarding tobacco use at regular intervals. Students also suggested that practice sessions regarding how to use information about tobacco in interactions with other students would be helpful.
[See Table 3 for supporting direct quotations from the interviews.]
Discussion
The increase in knowledge level reflected in the analysis of the pre- and post-tests is encouraging and demonstrates that the Tar Wars lesson plan is having the desired effect on increasing student knowledge about tobacco. Although the increase in total score of 1.28 points pre- to post-test is statistically significant, the educational significance is not known. We performed reliability calculations on the set of 14 questions in aggregate. The SEM) for the Tar Wars scale is 1.39, with the observed difference from pre- to post-test scores of 0.92 SEM. The SEM for this study comes close to a recommended criterion for meaningful change of 1 SEM.27 Three items recommended by the CDC to be included in youth tobacco prevention (cost, social norms, and advertising) showed larger changes. The qualitative portion of the evaluation also confirmed that new information is being presented by Tar Wars. In addition, both youth and teachers felt that although some of the material was already known before the class, the material was presented in an innovative manner and was valuable in reinforcing the tobacco-free message. Items showing change are included in the CDC recommendations for youth tobacco prevention.
The ceiling effect seen on several questions decreased the amount of positive change that was possible, both for the individual items and the overall knowledge scores. Some of these items are very important concepts for youth tobacco prevention that need to be reinforced repeatedly, and continued focus on these areas is certainly justified. The 3 questions showing ambiguous change seem to be written such that the intended correct answer is not clear. To decrease the test ceiling effect and avoid confusion on questions, test items and accompanying instruction could be reworded, which could result in a greater intervention effect. In addition, some revision of the Tar Wars lesson plan could be undertaken to introduce more new material that is not currently known by 5th grade students.
Although all students displayed a positive effect from the Tar Wars lesson plan, the slight differential effect between white students and minority students could indicate that the lesson plan is not affecting minority students quite as much as white students. The Tar Wars lesson plan is now available in a Spanish-language version that may change its effectiveness among this target population and may need to be assessed independently. This also could be explored further through such methodology as targeted focus groups with minority teachers and students to investigate the cultural appropriateness of the lesson plan.
The quantitative analysis has several limitations. First, pre- and post-tests could not be identified at the student level. Individual students were not identified in this study because of Institutional Review Board issues, cost, and time limitations. The study had state funding with a strict time limitation and schools were reluctant to individually identify students to preserve confidentiality. As a result of this, actual differences between specific students could not be analyzed. Instead, a more general analysis was done comparing groups of students at the school level. Pre- and post-tests showed no variance in demographics among the students, indicating a high probability of the tested populations representing the same group. Absenteeism would tend to produce the null response and decrease the shown effectiveness of the program. A future study using an ID code system to identify individual students would add to the study’s power.
A second limitation to the study is that no control group was used. Without a control group, the results should be considered as highly suggestive of an effect of the Tar Wars educational program, but not totally conclusive, because other sources of the observed changes are possible (if unlikely). For this study, it was decided not to evaluate intent to use tobacco or actual use rates because of the ceiling effect in children of this age given their low use rate and high stated intent to not use tobacco. In addition, the design did not allow assessment of the long-term effect of Tar Wars on student knowledge, attitudes, or behavior. Such a study would be difficult and somewhat expensive, but would provide more definitive support for the effectiveness of Tar Wars.
The qualitative portion of the evaluation provides further information supporting the positive educational impact of the Tar Wars program. New information is being presented in the Tar Wars sessions about core educational issues about tobacco, and other issues that may be already known are being reinforced in a positive manner. Students, teachers, and presenters emphasized the importance of repetition of anti-tobacco sessions and messages. Even though some of the material is already known before the session, repeated reminders of the key messages and placement of the information into an overall context of tobacco avoidance are important.28 The use of health professionals as presenters is perceived very positively, providing an authoritative outside source of information that adds to the impact of the session.
This study shows that Tar Wars produces significant change in the school instruction recommendation portions of the CDC’s Department of Adolescent and School Health 2000 Guidelines for School Programs to Prevent Tobacco Use, which recommends introduction of tobacco prevention education in elementary school.2 In addition, Tar Wars demonstrated significant change in 7 of 9 of the recommended knowledge portions of the K-12 instructional concepts of the 1994 CDC Guidelines for School Health Programs to Prevent Tobacco Use and Addiction.10 It is important to emphasize that school-based tobacco education is one part of a comprehensive approach to youth tobacco prevention activities that has been shown to decrease youth tobacco initiation.13–22
The Tar Wars lesson plan is effective in increasing students’ understanding about the short-term consequences of tobacco use, cost of tobacco use, truth of tobacco advertising, and peer norms. It can be considered an effective lesson plan as part of a comprehensive school-based strategy. Tar Wars meets CDC guidelines as one component of effective comprehensive youth tobacco prevention. Continued efforts to improve the lesson plan, would be appropriate, but major changes are not needed. Family physicians can play an essential role in educating our nation’s youth about the facts (and fiction) of tobacco use.
Acknowledgments
Special thanks are extended to the staff of Colorado Tar Wars and the CAFPF, who operated Colorado Tar Wars during the years of the study: Tina Goldstein, Director CO Tar Wars; Kimberly Cole, Coordinator CO Tar Wars; Teresa Schriener, Administrative Assistant CO Tar Wars; Raquel Alexander, MA, CAE, Executive Vice President CAFPF; and Liz Westerfield, CAFPF Grants Consultant.
Notes
This article was externally peer-reviewed.
Funding: This work was supported in part by funding from the Colorado Department of Public Health and Environment’s State Tobacco Education and Prevention Partnership.
Previous presentation: Some of the data in this article was presented at the 17th WONCA World Conference of Family Doctors on 14 October 2004 in Orlando, FL.
Conflict of interest: JJC is the cofounder and former president of Tar Wars, which is now owned and operated by the AAFP.
References
Centers for Disease Control and Prevention. Reducing tobacco use: A report of the Surgeon General-2000. Available at: "(www.cdc.gov/tobacco/sgr/sgr_2000/aag2cpy.pdf"). Accessed December 14, 2005.
Centers for Disease Control and Prevention/Division of Adolescent and School Health. Guidelines for school health programs to prevent tobacco use: Summary. Available at: "(www.cdc.gov/HealthyYouth/tobacco/pdf/summary.pdf"). Accessed December 14, 2005.
Centers for Disease Control. Cigarette use among high school students-United States, 1991–2003. 53. 2004; 53: 499–522. Available at: "(www.cdc.gov/tobacco/research_data/youth/mm5323_intro.htm"). Accessed December 14, 2005.
Centers for Control Disease. Preventing tobacco use among young people: A report of the Surgeon General, 1994. Atlanta, GA: Public Health Service, Centers for Disease Control and Prevention, Office on Smoking and Health, 1994.
Kaufman NJ, Castrucci BC, Mowery PD, Gerlach KK, Emont S, Orleans CT. Predictors of change on the smoking uptake continuum among adolescents. Arch Pediatr Adolesc Med 2002; 156: 581–7.
Evans N, Farkas A, Gilpin E, Berry C, Pierce JP. Influence of tobacco marketing and exposure to smokers on adolescent susceptibility to smoking. J Natl Cancer Inst 1995; 87: 1538–45.
Pierce JP, Choi WS, Gilpin EA, Farkas AJ, Berry CC. Tobacco industry promotion of cigarettes and adolescent smoking. JAMA 1998; 279: 511–5.
Simons-Morton B, Crump AD, Haynie DL, Saylor KE, Eitel P, Yu K. Psychosocial, school, and parent factors associated with recent smoking among early-adolescent boys and girls. Prev Med 1999; 28: 138–48.
Centers for Disease Control and Prevention. Best Practices for Comprehensive Tobacco Control Programs–August 1999. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health, 1999.
Anonymous. Guidelines for school health programs to prevent tobacco use and addiction. Centers for Disease Control and Prevention. MMWR Recomm Rep 1994; 43: 1–18.
Lynch BS, Bonnie RJ, Institute of Medicine (U.S.), Committee on Preventing Nicotine Addiction in Children and Youths. Growing Up Tobacco Free: Preventing Nicotine Addiction in Children and Youths. Washington DC: National Academy Press, 1994.
Dent CW, Sussman S, Stacy AW, Craig S, Burton D, Flay BR. Two-year behavior outcomes of project towards no tobacco use. J Consult Clin Psychol 1995; 63: 676–7.
Abt Associates, Inc. Independent Evaluation of the Massachusetts Tobacco Control Program, Seventh Annual Report: January 1994 to June 2000.
Anonymous. Massachusetts Youth Risk Behavior Survey: 2001; National Youth Risk Behavior Survey.
Florida Department of Health. 2002 Florida Youth Tobacco Survey, Volume 5, Report 1. 2003 May.
Oregon Department of Health and Human Services. Make Oregon Healthier: Saving Lives and Saving Dollars. 2001–2003 Program Report. Tobacco Prevention and Education Program. Available at: "(http://oregon.gov/DHS/ph/tobacco/arpt2001/arpt2001.pdf"). Accessed December 14, 2005.
Centers for Control Disease. Tobacco use Among Adults- Arizona, 1996–1999. 50. 2001; 50: 402–6. Available at: "(www.cdc.gov/mmwr/preview/mmwrhtml/mm5020a2.htm"). Accessed December 14, 2005.
University of Texas- Houston. Texas Tobacco Prevention Initiative: Media Campaign and Community Program. Effects Among Children and Adults, January 2001.
Anonymous. Maine 2003 Youth Risk Behavior Survey, Maine Department of Human Services, October, 2003.
Anonymous. Effectiveness of school-based programs as a component of a statewide tobacco control initiative–Oregon, 1999–2000. MMWR Morb Mortal Wkly Rep 2001; 50: 663–6.
Flynn BS, Worden JK, Secker-Walker RH, Pirie PL, Badger GJ, Carpenter JH. Long-term responses of higher and lower risk youths to smoking prevention interventions. Prev Med 1997; 26: 389–94.
Lantz PM, Jacobson PD, Warner KE, Wasserman J, Pollack HA, Berson J, et al. Investing in youth tobacco control: a review of smoking prevention and control strategies. Tob Control 2000; 9: 47–63.
Cain JJ, Dudley TE, Wilkerson MK. Tar Wars: A community-based tobacco education project. J Fam Pract 1992; 34: 267–8.
Mahoney MC, Costley CM, Cain J, Zaiger D, McMullen S. School nurses as advocates for youth tobacco education programs: The TAR WARS experience. J Sch Health 1998; 68: 339–41.
Mahoney MC, Stengel B, McMullen S, Brown S. Evaluation of a youth tobacco education program: student, teacher, and presenter perspectives. J Sch Nurs 2000; 16: 16–21.
Miller W, Crabtree B. Clinical research: A multimethod typology and qualitative roadmap. Miller W, Crabtree B, eds. Doing Qualitative Research. Thousand Oaks, CA: Sage Publications, Inc., 1999; 3–30.
Wyrwich KW, Tierney WM, Wolinsky FD. Further Evidence Supporting an SEM-Based Criterion for Identifying Meaningful Intra-Individual Changes in Health-Related Quality of Life. J Clin Epidemiol 1999; 52: 861–73.
Reinert B, Carver V, Range LM. Anti-tobacco messages from different sources make a difference with secondary school students. J Public Health Manag Pract 2004; 10: 518–23.(Jeffrey J. Cain, MD, W. Perry Dickinson,)
Department of Family Medicine, The Children’s Hospital, Denver, CO
Abstract
Background: Tar Wars is a national school-based tobacco-free education program operated by the American Academy of Family Physicians. The Tar Wars lesson uses an interactive 45-min session taught by volunteer family physicians in 4th- and 5th-grade classrooms and focuses on the short-term image-based consequences of tobacco use. In this study, we evaluated the effectiveness of the Tar Wars program in Colorado with both quantitative and qualitative measures.
Methods: Students participating in the quantitative evaluation were tested before and after a Tar Wars teaching session using a 14-question test covering the short-term and image-based consequences of tobacco use, cost of smoking, tobacco advertising, and social norms of tobacco use. Qualitative evaluation of the program included guided telephone interviews and focus groups with participating students, teachers, and presenters.
Results: Quantitative evaluation showed statistically significant improvement in correct responses for the 14 questions measured with an average increase in correct responses from 8.95 to 10.23. Three areas recommended by the Centers for Disease Control (CDC) for youth tobacco prevention showed greater change in correct responses, including cost of smoking, truth of tobacco advertising, and peer norms of tobacco use. Qualitative evaluation found that the overall message of the session was well received, that previously known tobacco information was reinforced by its presentation in a novel format, and that new information learned included cost of smoking, truth of tobacco advertising, and peer norms of tobacco use.
Conclusions: The Tar Wars lesson plan is effective in increasing students’ understanding about the short-term consequences of tobacco use, cost of tobacco use, truth of tobacco advertising, and peer norms. Tar Wars meets the CDC guidelines as one component of effective comprehensive youth tobacco prevention.
Tobacco remains the no. 1 preventable cause of death in the United States, causing >400,000 deaths annually.1 Each day in the United States, approximately 4400 youths aged 12 to 17 years try their first cigarette,2 with 21.9% of US high school students using tobacco regularly.3 Nine of 10 current adult smokers started their habit before the age of 19 years.4 Major influences on youth tobacco initiation continue to be tobacco advertising5–7 and social norms.8
The 2000 Surgeon General’s Report recommends comprehensive tobacco prevention activities to reduce tobacco use, including community interventions that include school-based education.1 Because many students start using tobacco before high school, the Centers for Disease Control (CDC) Best Practices recommend that tobacco use prevention begin in elementary school and continue through middle and high school.9 The CDC Department of Adolescent and School Health’s Tobacco Prevention Guidelines specifically recommend school-based instruction about the short- and long-term negative physiologic and social consequences of tobacco use, social influences on tobacco use, peer norms regarding tobacco use, and refusal skills.10 Instructional programs using these components have been shown to significantly reduce smoking prevalence for up to 5 years after program completion.11, 12
Comprehensive statewide tobacco prevention programs have been shown to effectively reduce youth tobacco use13–19 and have shown school-based programs to be an effective component of statewide tobacco control.20, 21 Meta-analysis of youth tobacco prevention programs suggests that those programs focusing on social influence resistance models are more effective than programs designed to arouse concern or fear by focusing on death and other health risks.22
Tar Wars is an in-school, tobacco-free education program owned and operated by the American Academy of Family Physicians (AAFP). Tar Wars was designed in 1988 in Denver, CO, by Doctors Ought to Care (DOC) and the Hall of Life at the Denver Museum of Nature and Science.23 Tar Wars ownership was acquired by the AAFP in 2000, is currently taught in 50 states and 14 countries, and has reached 8 million children. Tar Wars is a lesson plan intended to be one component in a comprehensive approach to youth tobacco prevention. Previously published literature reports on Tar Wars have included descriptions of the program23, 24 and program perspective of involved students, teachers, and presenters.25 This study provides a qualitative and quantitative examination of the impact and effects of the Tar Wars program in Colorado elementary schools.
Methods
During 2001 and 2002, the Colorado Tar Wars program was operated by the Colorado Academy of Family Physicians Foundation (CAFPF) with a grant from the Colorado Department of Public Health and Environment, State Tobacco Education and Prevention Partnership (STEPP). Researchers from the Department of Family Medicine at the University of Colorado Health Sciences Center independent of the CAFPF and Tar Wars conducted this evaluation of the program, which was approved by STEPP. Because of constraints of the STEPP grant, the study was limited to implementation and evaluation of the Tar Wars program during one academic year.
This study evaluates the Tar Wars lesson program as implemented in Colorado during the 2001–2002 academic year. The Tar Wars lesson plan evaluated is offered free through the AAFP Web site (www.tarwars.org) and includes 3 components: a preactivity given by the classroom teacher in the days before the guest speaker, a classroom session led by volunteer family physicians and health educators, and the poster contest coordinated by the class teacher. The preactivity’s objective is to teach students that, contrary to their perception, tobacco users are in the minority. The classroom session includes 5 activities: short-term effects of tobacco use (bad breath, smelly clothes, yellow teeth, coughing, ashes, etc), demonstration of the physical effect of decreased lung capacity (breathing-through-straw exercise), cost of tobacco use (per week, month, year), reasons smokers give for tobacco use (contrasted with actual short-term effects), and an analysis of tobacco advertising. The poster contest challenges students to illustrate their integration of the Tar Wars lessons by the creation of a poster showing the positive effects of not using tobacco.
Schools were recruited by mailing invitations to participate in Tar Wars to all 5th-grade classrooms in the state. Family physicians were recruited by a mailing sent to all members of the CAFP. Schools were matched geographically with family physicians by the CAFPF Tar Wars staff. The evaluation included the first 68 schools matched with speakers, with the goal of approaching 3000 students. Speaker and teacher training consisted of providing each with copies of the Tar Wars program guide as offered by the AAFP. The Tar Wars lesson plan guides teachers and presenters on each activity, including an objective, time requirement, suggested speaker comments, and activity instructions. The session is designed to be interactive with a high level of student participation.
For the quantitative portion of this study, surveys containing 14 questions regarding knowledge about smoking were administered before and after the Tar Wars session to students of 68 elementary schools in Colorado. Copies of the pre- and post-tests are included in Figures 1 and 2. Questions directly relating to lesson plan content were obtained from the AAFP’s unpublished Tar Wars survey, part of the package of Tar Wars materials. Minor changes to the questions were made by the CAFPF Tar Wars Committee and the Department of Family Medicine for clarity and to better reflect the content of the session. Fifth grade students ranging in age from 9 to 13 years completed the pre- and post-tests. This analysis attempts to determine whether there were any changes in knowledge about tobacco on the responses to the pre- and post-tests. For comparison purposes, all questions from the pretest were also asked on the post-test. Pretests were administered approximately 1 week before the intervention, and post-tests were administered 2 to 4 days after the Tar Wars presentation. To preserve student confidentiality, unique identifiers were not used in the data collection. As a result, comparisons over time could not be made at the individual student level, but school identification was retained, allowing comparisons over time across the school groups.
The 2 test of association was used to determine demographic differences between the pre- and post-test respondent population. Because multiple 2 tests were performed, statistical significance was determined at the level of 0.01. Assessment of differences between correct responses to the 14 items and average composite score (pre- and post-test) were performed using multi-level modeling, controlling for age, race, ethnicity, gender; whether there was a smoker in the household; and whether the respondent was currently using tobacco. The interclass correlation (ICC) was calculated to determine whether student composite scores vary between schools as well as within schools. General linear mixed models (Proc Mixed) with average composite score as the outcome were used to extend the traditional linear regression model to accommodate the hierarchical structure. Because we were unable to retain student identifiers, the study design was analyzed as a nested cross-sectional design, with students nested within schools at 2 distinct time points, adjusting for the tendency for students within the same school to respond more similarly than students in different schools. Variables that were significant in the model at the level of 0.10 were tested for possible interaction effects with the test (pre- vs post-test) variable. Current tobacco use was not considered because the total number of tobacco users in the sample was very small. Interactions significant at the level of 0.05 were retained in the model.
The qualitative evaluation used 2 primary methods of data collection: semi-structured interviews and focus group discussions. Semistructured, confidential telephone interviews were conducted with presenters and teachers who participated in Tar Wars during the 2001–2002 school year. In addition, evaluation forms filled out by presenters and classroom teachers after the presentations were reviewed qualitatively, focusing on the responses to the open-ended probes asking for suggestions for improvement in the program.
The CAFPF provided the evaluation team with lists of presenters, teachers, and youth educators for whom they had current contact information. We selected multiple names from the list to cover a wide geographic area within the state—both urban and rural—and scheduled interviews only with those who returned our calls and consented to an interview. The interviews asked about their experiences, reactions from students, challenges, and how the program could be improved. These interviews were audiotaped and transcribed.
Focus groups with students who participated in Tar Wars were designed to elicit responses around what students remembered from the presentation, what they liked or disliked, how they used or might use what they learned, and what ideas they have for helping other kids learn about tobacco. Along with the interview guide, additional probes were used to elicit more detailed responses where necessary. Because of funding and time limitations of the evaluation, the CAFPF contacted classroom teachers directly in the Denver metropolitan area to set up the student focus groups. In addition, because of the age of the students and possible negative reactions to audiotaping the interviews, the evaluation team decided not to audiotape focus groups. Both the facilitator and co-facilitator kept notes, including direct quotations when possible.
The transcripts and typed notes were analyzed using ATLAS-ti qualitative data analysis software application. The analyst used an "editing" style of analysis.26 The editing style presumes no pre-existing theories or hypotheses. Using this style, the researcher starts with a "blank slate" and allows the data to guide the discovery of evolving patterns or themes.
In general, we identified the most significant themes and issues that reflected views expressed by the key informants. The emergent themes were discussed among co-facilitators and the CAFPF staff to confirm or disconfirm their meaning and relevance. Results from paper surveys by classroom teachers were also reviewed to verify results from the teacher interviews conducted by the evaluation team. Salient themes discussed below generally reflect the experience of multiple individuals. Some views were expressed by one or several individuals that were not shared by others, yet reflect important contradictory or insightful perspectives.
Results
Quantitative Analysis
A total of 2926 students completed the pretest, and 2766 students completed the post-test and were included in this analysis. The pre- and post-Tar Wars respondent populations were not significantly different (P > .05) in terms of their demographics (Table 1). The sample of schools was diverse in terms of geographic spread across Colorado and represented metropolitan, urban/suburban, rural, and frontier communities. The demographics correspond generally to Colorado demographic data. The unconditional ICC correlation for students within schools was 0.08, with a conditional ICC of 0.06 (variance component Final Model (school): 0.2131, P < .0001), suggesting that hierarchical analysis would be necessary to account for the clustering of students within schools.
Analysis of the pre- and post-tests indicated that the tobacco knowledge of students increased from pre- to post-test, in general as well as across a wide variety of focus areas. Respondents’ average scores were significantly higher for the post-test than the pretest (P < .0001). This finding remained true after controlling for age, gender, race, ethnicity; whether the respondent lives with a tobacco user; whether the respondent is a current tobacco user; and clustering of students within schools. Students increased their knowledge score by an average of 1.28 correct answers on the 14 items after the intervention, with an average overall knowledge score increase from 8.95 correct responses before the Tar Wars session to 10.23 correct responses after the Tar Wars lesson. Knowledge scores showed an increase in correct responses in all 14 of the pre- and post-test responses (Table 2), with the proportion of correct responses showing significant increase between tests (P < .01).
Significant predictors of average knowledge scores included race (P = .0003), ethnicity (P < .0001), and current tobacco use by the respondent (P < .0001). For these predictors, knowledge scores are reduced by one-half a point (race), two-thirds of a point (ethnicity), and 1.4 points (current tobacco use). Significant interactions included test by ethnicity (P = .0007) as well as race by test (P = .0113). The average increase in tobacco knowledge score for minority students (non-white and Hispanic/Latino respondents) was lower than the increase in score of white non-Hispanic/Latino students. The average increase in knowledge score for minority students was 1.08 (Hispanic) and 1.10 (nonwhite students). White non-Hispanic students’ average knowledge scores increased approximately 1.50 points between tests.
The distribution of question responses tended to cluster around 3 categories:
Items for which a statistical significant improvement occurred, but a high percentage of respondents already knew the correct answers before the intervention: more than 75% of the population selected the correct response at pretest for each question. While there was a statistically significant improvement in the percentage of correct responses between pre- and post-tests, a ceiling effect exists.
For 7 of the 14 knowledge questions, the post-test correct responses increased between 4% and 10% in a significant way, indicating additional learning. These items included: "Smoking causes bad breath," "Smokers have yellow teeth," "Cigarettes that contain low tar are safe," "Smokers have clean-smelling clothes," "Smoking can decrease the amount of air that gets into your lungs," "People who use tobacco are cool, sexy, and popular," and "Some people use tobacco because their friends do."
Items that reflect a high degree of newly acquired tobacco knowledge: questions where the proportion of correct responses were initially low pretest and increased by 10% to 20%.
The correct responses to 3 of the fourteen knowledge questions were not as well known before the Tar Wars intervention, but were identified correctly by a larger percentage of the post-test respondent population. These questions included: "Smoking a pack of cigarettes each day for a year would cost at least several hundred dollars," "Advertisers tell the truth about tobacco use," and "What percentage of kids your age do not use tobacco every week."
Items with ambiguous change: questions where some learning may have occurred, but did not seem to be as effective across all responses.
These included "Smoking is relaxing," "Tobacco companies can use television as a way of advertising," and "G-rated movies are used by tobacco companies to sell tobacco."
For each of these questions there was a statistical association between pre- and post-test responses. However, even though response increases occurred between the correct responses, increases also occurred with either the incorrect responses or the "don’t know" responses.
Qualitative Analysis
We completed semistructured individual interviews (10–20 min in length) with 9 presenters, 8 teachers, and 3 youth educators. We completed 5 focus group discussions, with group sizes ranging from 7 to 15 participants. By the completion of the 5 focus group discussions, it seemed that saturation had been achieved, with few if any new topics of substance arising.
Students, presenters, and teachers whom we interviewed agreed that they liked the Tar Wars presentations and activities and that new learning occurred. Presenters and teachers reported that students were attentive and interested and reacted positively to the information, presentation style, and having someone from outside the school (especially physicians) present the information. They thought that the content and length were appropriate and that the use of hands-on activities and visual aids was effective at engaging the students. The teachers whom we interviewed agreed that Tar Wars integrates well with other health education or science lessons.
The students responded most strongly to the central activities in the lesson plan and recalled the information about tobacco use: advertising, effects on physical appearance, percentages of those who smoke, and the difficulty of breathing for smokers. The students thought that the Tar Wars program presented a different kind of message about tobacco than some of the other programs that they had experienced. For many students, the information about the cost of smoking, the percentage of youth and adults who smoke, and how tobacco is advertised also was new information.
Most students, presenters, and teachers acknowledged that students already knew that using tobacco is harmful, with long-term health effects such as cancer. However, they thought that the details about the short-term effects and other information about tobacco (eg, cost, ingredients, and advertising) were new information that helped them to understand why they should not use tobacco. Many students emphasized the importance of hearing these messages repeatedly and suggested having various sessions regarding tobacco use at regular intervals. Students also suggested that practice sessions regarding how to use information about tobacco in interactions with other students would be helpful.
[See Table 3 for supporting direct quotations from the interviews.]
Discussion
The increase in knowledge level reflected in the analysis of the pre- and post-tests is encouraging and demonstrates that the Tar Wars lesson plan is having the desired effect on increasing student knowledge about tobacco. Although the increase in total score of 1.28 points pre- to post-test is statistically significant, the educational significance is not known. We performed reliability calculations on the set of 14 questions in aggregate. The SEM) for the Tar Wars scale is 1.39, with the observed difference from pre- to post-test scores of 0.92 SEM. The SEM for this study comes close to a recommended criterion for meaningful change of 1 SEM.27 Three items recommended by the CDC to be included in youth tobacco prevention (cost, social norms, and advertising) showed larger changes. The qualitative portion of the evaluation also confirmed that new information is being presented by Tar Wars. In addition, both youth and teachers felt that although some of the material was already known before the class, the material was presented in an innovative manner and was valuable in reinforcing the tobacco-free message. Items showing change are included in the CDC recommendations for youth tobacco prevention.
The ceiling effect seen on several questions decreased the amount of positive change that was possible, both for the individual items and the overall knowledge scores. Some of these items are very important concepts for youth tobacco prevention that need to be reinforced repeatedly, and continued focus on these areas is certainly justified. The 3 questions showing ambiguous change seem to be written such that the intended correct answer is not clear. To decrease the test ceiling effect and avoid confusion on questions, test items and accompanying instruction could be reworded, which could result in a greater intervention effect. In addition, some revision of the Tar Wars lesson plan could be undertaken to introduce more new material that is not currently known by 5th grade students.
Although all students displayed a positive effect from the Tar Wars lesson plan, the slight differential effect between white students and minority students could indicate that the lesson plan is not affecting minority students quite as much as white students. The Tar Wars lesson plan is now available in a Spanish-language version that may change its effectiveness among this target population and may need to be assessed independently. This also could be explored further through such methodology as targeted focus groups with minority teachers and students to investigate the cultural appropriateness of the lesson plan.
The quantitative analysis has several limitations. First, pre- and post-tests could not be identified at the student level. Individual students were not identified in this study because of Institutional Review Board issues, cost, and time limitations. The study had state funding with a strict time limitation and schools were reluctant to individually identify students to preserve confidentiality. As a result of this, actual differences between specific students could not be analyzed. Instead, a more general analysis was done comparing groups of students at the school level. Pre- and post-tests showed no variance in demographics among the students, indicating a high probability of the tested populations representing the same group. Absenteeism would tend to produce the null response and decrease the shown effectiveness of the program. A future study using an ID code system to identify individual students would add to the study’s power.
A second limitation to the study is that no control group was used. Without a control group, the results should be considered as highly suggestive of an effect of the Tar Wars educational program, but not totally conclusive, because other sources of the observed changes are possible (if unlikely). For this study, it was decided not to evaluate intent to use tobacco or actual use rates because of the ceiling effect in children of this age given their low use rate and high stated intent to not use tobacco. In addition, the design did not allow assessment of the long-term effect of Tar Wars on student knowledge, attitudes, or behavior. Such a study would be difficult and somewhat expensive, but would provide more definitive support for the effectiveness of Tar Wars.
The qualitative portion of the evaluation provides further information supporting the positive educational impact of the Tar Wars program. New information is being presented in the Tar Wars sessions about core educational issues about tobacco, and other issues that may be already known are being reinforced in a positive manner. Students, teachers, and presenters emphasized the importance of repetition of anti-tobacco sessions and messages. Even though some of the material is already known before the session, repeated reminders of the key messages and placement of the information into an overall context of tobacco avoidance are important.28 The use of health professionals as presenters is perceived very positively, providing an authoritative outside source of information that adds to the impact of the session.
This study shows that Tar Wars produces significant change in the school instruction recommendation portions of the CDC’s Department of Adolescent and School Health 2000 Guidelines for School Programs to Prevent Tobacco Use, which recommends introduction of tobacco prevention education in elementary school.2 In addition, Tar Wars demonstrated significant change in 7 of 9 of the recommended knowledge portions of the K-12 instructional concepts of the 1994 CDC Guidelines for School Health Programs to Prevent Tobacco Use and Addiction.10 It is important to emphasize that school-based tobacco education is one part of a comprehensive approach to youth tobacco prevention activities that has been shown to decrease youth tobacco initiation.13–22
The Tar Wars lesson plan is effective in increasing students’ understanding about the short-term consequences of tobacco use, cost of tobacco use, truth of tobacco advertising, and peer norms. It can be considered an effective lesson plan as part of a comprehensive school-based strategy. Tar Wars meets CDC guidelines as one component of effective comprehensive youth tobacco prevention. Continued efforts to improve the lesson plan, would be appropriate, but major changes are not needed. Family physicians can play an essential role in educating our nation’s youth about the facts (and fiction) of tobacco use.
Acknowledgments
Special thanks are extended to the staff of Colorado Tar Wars and the CAFPF, who operated Colorado Tar Wars during the years of the study: Tina Goldstein, Director CO Tar Wars; Kimberly Cole, Coordinator CO Tar Wars; Teresa Schriener, Administrative Assistant CO Tar Wars; Raquel Alexander, MA, CAE, Executive Vice President CAFPF; and Liz Westerfield, CAFPF Grants Consultant.
Notes
This article was externally peer-reviewed.
Funding: This work was supported in part by funding from the Colorado Department of Public Health and Environment’s State Tobacco Education and Prevention Partnership.
Previous presentation: Some of the data in this article was presented at the 17th WONCA World Conference of Family Doctors on 14 October 2004 in Orlando, FL.
Conflict of interest: JJC is the cofounder and former president of Tar Wars, which is now owned and operated by the AAFP.
References
Centers for Disease Control and Prevention. Reducing tobacco use: A report of the Surgeon General-2000. Available at: "(www.cdc.gov/tobacco/sgr/sgr_2000/aag2cpy.pdf"). Accessed December 14, 2005.
Centers for Disease Control and Prevention/Division of Adolescent and School Health. Guidelines for school health programs to prevent tobacco use: Summary. Available at: "(www.cdc.gov/HealthyYouth/tobacco/pdf/summary.pdf"). Accessed December 14, 2005.
Centers for Disease Control. Cigarette use among high school students-United States, 1991–2003. 53. 2004; 53: 499–522. Available at: "(www.cdc.gov/tobacco/research_data/youth/mm5323_intro.htm"). Accessed December 14, 2005.
Centers for Control Disease. Preventing tobacco use among young people: A report of the Surgeon General, 1994. Atlanta, GA: Public Health Service, Centers for Disease Control and Prevention, Office on Smoking and Health, 1994.
Kaufman NJ, Castrucci BC, Mowery PD, Gerlach KK, Emont S, Orleans CT. Predictors of change on the smoking uptake continuum among adolescents. Arch Pediatr Adolesc Med 2002; 156: 581–7.
Evans N, Farkas A, Gilpin E, Berry C, Pierce JP. Influence of tobacco marketing and exposure to smokers on adolescent susceptibility to smoking. J Natl Cancer Inst 1995; 87: 1538–45.
Pierce JP, Choi WS, Gilpin EA, Farkas AJ, Berry CC. Tobacco industry promotion of cigarettes and adolescent smoking. JAMA 1998; 279: 511–5.
Simons-Morton B, Crump AD, Haynie DL, Saylor KE, Eitel P, Yu K. Psychosocial, school, and parent factors associated with recent smoking among early-adolescent boys and girls. Prev Med 1999; 28: 138–48.
Centers for Disease Control and Prevention. Best Practices for Comprehensive Tobacco Control Programs–August 1999. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health, 1999.
Anonymous. Guidelines for school health programs to prevent tobacco use and addiction. Centers for Disease Control and Prevention. MMWR Recomm Rep 1994; 43: 1–18.
Lynch BS, Bonnie RJ, Institute of Medicine (U.S.), Committee on Preventing Nicotine Addiction in Children and Youths. Growing Up Tobacco Free: Preventing Nicotine Addiction in Children and Youths. Washington DC: National Academy Press, 1994.
Dent CW, Sussman S, Stacy AW, Craig S, Burton D, Flay BR. Two-year behavior outcomes of project towards no tobacco use. J Consult Clin Psychol 1995; 63: 676–7.
Abt Associates, Inc. Independent Evaluation of the Massachusetts Tobacco Control Program, Seventh Annual Report: January 1994 to June 2000.
Anonymous. Massachusetts Youth Risk Behavior Survey: 2001; National Youth Risk Behavior Survey.
Florida Department of Health. 2002 Florida Youth Tobacco Survey, Volume 5, Report 1. 2003 May.
Oregon Department of Health and Human Services. Make Oregon Healthier: Saving Lives and Saving Dollars. 2001–2003 Program Report. Tobacco Prevention and Education Program. Available at: "(http://oregon.gov/DHS/ph/tobacco/arpt2001/arpt2001.pdf"). Accessed December 14, 2005.
Centers for Control Disease. Tobacco use Among Adults- Arizona, 1996–1999. 50. 2001; 50: 402–6. Available at: "(www.cdc.gov/mmwr/preview/mmwrhtml/mm5020a2.htm"). Accessed December 14, 2005.
University of Texas- Houston. Texas Tobacco Prevention Initiative: Media Campaign and Community Program. Effects Among Children and Adults, January 2001.
Anonymous. Maine 2003 Youth Risk Behavior Survey, Maine Department of Human Services, October, 2003.
Anonymous. Effectiveness of school-based programs as a component of a statewide tobacco control initiative–Oregon, 1999–2000. MMWR Morb Mortal Wkly Rep 2001; 50: 663–6.
Flynn BS, Worden JK, Secker-Walker RH, Pirie PL, Badger GJ, Carpenter JH. Long-term responses of higher and lower risk youths to smoking prevention interventions. Prev Med 1997; 26: 389–94.
Lantz PM, Jacobson PD, Warner KE, Wasserman J, Pollack HA, Berson J, et al. Investing in youth tobacco control: a review of smoking prevention and control strategies. Tob Control 2000; 9: 47–63.
Cain JJ, Dudley TE, Wilkerson MK. Tar Wars: A community-based tobacco education project. J Fam Pract 1992; 34: 267–8.
Mahoney MC, Costley CM, Cain J, Zaiger D, McMullen S. School nurses as advocates for youth tobacco education programs: The TAR WARS experience. J Sch Health 1998; 68: 339–41.
Mahoney MC, Stengel B, McMullen S, Brown S. Evaluation of a youth tobacco education program: student, teacher, and presenter perspectives. J Sch Nurs 2000; 16: 16–21.
Miller W, Crabtree B. Clinical research: A multimethod typology and qualitative roadmap. Miller W, Crabtree B, eds. Doing Qualitative Research. Thousand Oaks, CA: Sage Publications, Inc., 1999; 3–30.
Wyrwich KW, Tierney WM, Wolinsky FD. Further Evidence Supporting an SEM-Based Criterion for Identifying Meaningful Intra-Individual Changes in Health-Related Quality of Life. J Clin Epidemiol 1999; 52: 861–73.
Reinert B, Carver V, Range LM. Anti-tobacco messages from different sources make a difference with secondary school students. J Public Health Manag Pract 2004; 10: 518–23.(Jeffrey J. Cain, MD, W. Perry Dickinson,)